Proposed Changes

Previously we would only add peer transfers to the transfer table in vine_manager_put_url after the task had been scheduled.

If a task had many input files, such as temporaries that reside on one worker, we would be happy to assign them all to be retrieved from the same worker in the scheduling stage, greater than the limit for transfers from a single source, since the task has not been sent yet and we were not incrementing the source_in_use count.

In some fashion we need to increment the source_in_use count while considering options for peer transfers. It is a bit difficult since there are a few scenarios we give up scheduling so any counting we did in the scheduling attempt needs to get reset.

Merge Checklist

The following items must be completed before PRs can be merge.
Check these off to verify you have completed all steps.

• make test Run local tests prior to pushing.
• make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
• make lint Run lint on source code prior to pushing.
• Manual Update Update the manual to reflect user-visible changes.
• Type Labels Select a github label for the type: bugfix, enhancement, etc.
• Product Labels Select a github label for the product: TaskVine, Makeflow, etc.
• PR RTM Mark your PR as ready to merge.